Color information is an important feature for many vision algorithms including color correction, image retrieval and tracking. Color correction methods are used to compensate for illumination conditions. In human perception such correction is called color constancy, which is the ability to perceive a relatively constant color for an object even under varying illumination. Most computer methods are pixel-based, correcting an image so that the image's statistics satisfy assumptions. One such assumption is the average intensity of the scene under neutral light being achromatic. Another assumption is that for a given illuminant, there is a limited number of expected colors in a real-world scene.
Various schemes have been proposed to use features including higher order derivatives or homogeneous color regions. These features are chosen based on their likelihood to best characterize the illuminant color and ignore the specific color of the objects in the scene. For example, higher order derivatives are used based on the assumption that the average of reflectance differences in a scene is achromatic. As another example, homogeneously colored segments can be used to catalog the colors that appear in a scene, thereby reducing the influence of a single large region.
However, none of the existing methods account for the fact that even at the level of the individual pixels, the reliability of the color information varies.
Color constancy methods have been categorized into three major groups (static, gamut-mapping, and learning-based) and extensively evaluated by Gijsenij et al. See E. A. Gijsenij, T. Gevers, and J. van de Wiejer, “Computational Color Constancy: Survey and Experiments,” IEEE Trans. On Image Processing, Vol 20, No. 9, September 2011.
These methods are an important precursor to improve algorithms which rely on color such as image retrieval, matching color across cameras and long term background modeling for surveillance. The typical methodology, whether based on learning or not, and regardless of the underlying assumptions, still rely on global image statistics.
Preprocessing for color correction often involves local averaging or Gaussian smoothing. This serves to reduce noise and has been shown to be beneficial for color correction. See K. Barnard, L. Martin, A. Coath, and B. Funt, “A comparison of computational color constancy algorithms—part ii: experiments with image data,” IEEE Trans. on Image Processing, vol. 11, no. 9, pp 985-996, September 2002. See, also, J. van de Weijer, Th. Gebers, A. Gijsenij, “Edge-based color constancy,” IEEE Trans. on Image Processing, vol. 16, no. 9, September 2007.
However, this type of noise reduction, while it may improve performance overall, introduces artifacts along high gradients and potentially ignores relevant information.
In earlier work in color histograms, methods are based on one of the perceptually uniform color spaces. Several investigators have concluded that the features derived from perceptually uniform space are in many ways optimal for color image retrieval. See D. Borhesani et al., “Color Features Performance Comparison for Image Retrieval,” Image Analysis and Processing—ICIAP 2009, Lecture Notes in Computer Science, Springer Berlin/Heidelberg, p 902-210, August 2009.
In order to avoid instability along the gray axis in hue-based spaces such HSV/HSB/HIS: (i) a weighting system was developed (see J. van de Weijer, T. Gevers, & A. Bagdanov, “Boosting Color Saliency in Image Feature Detection,” IEEE Trans. Pattern Analysis and Machine Intelligence (PAMI), Vol 28, No. 1, p 150-156, 2006); (ii) a non-uniform binning system was developed (see Z. Lei, et al., “A CBIR Method Based on Color-Spatial Feature,” Proc. IEEE Region 10 Annual International Conference 1999 (TENCON'99), Cheju, Korea. p 166-169, 1999; and (iii) a chromatic/achromatic splitting method by was developed (see L. Brown, “Example-based Color Vehicle Retrieval for Surveillance,” IEEE MMSS Workshop, Boston, Mass. September 2010).
In spite of such technology, it remains the case that hue information becomes unreliable near the gray axis: the transformation to hue-based spaces is ill-conditioned near the gray axes and the noise inherent in the raw RGB images is therefore amplified.